Investigating the Validity of Botometer-Based Social Bot Studies

The idea that social media platforms like Twitter are inhabited by vast numbers of bots has become widely accepted in recent years. Social assumed to be automated accounts operated malicious actors with the goal manipulating public opinion. They credited ability produce content autonomously and interact human users. bot activity been reported many different political contexts, including U.S. presidential elections, discussions about migration, climate change, COVID-19. However, relevant publications either use crude questionable heuristics discriminate between supposed humans or—in majority cases—fully rely on output automatic detection tools, most commonly Botometer. In this paper, we point out a fundamental theoretical flaw widely-used study design for estimating prevalence bots. Furthermore, empirically investigate validity peer-reviewed Botometer-based studies closely systematically inspecting hundreds had counted as We were unable find single bot. Instead, found mostly undoubtedly users, them using an inconspicuous unremarkable fashion without slightest traces automation. conclude claiming prevalence, properties, or influence based Botometer have, reality, just investigated false positives artifacts approach.